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Abstract 



The Kamiokande II and 1MB data on contained events induced by atmospheric 

neutrinos exhibit too low a ratio of muons to electrons, which has been interpreted 

■ as a possible indication of neutrino oscillations. At the same time, the recent data on 

i upward-going muons in underground detectors have shown no evidence for neutrino 

I oscillations, strongly limiting the allowed region of oscillation parameter space. In 

this paper we confront different types of neutrino oscillation hypotheses with the 

^ , experimental results. The matter effects in <-> Vf, and in ^ ^sterile oscillations 

Q ' are discussed and shown to affect significantly the upward-going muons. 

(N ' 
m 



Several times signals of neutrino oscillations have been claimed in the past. The 
^ ' evidence from accelerator and reactor experiments has meanwhile evaporated [0], leaving 

■ the observed solar neutrino deficit as the only possible indication. 



Recently, however, another hint of neutrino oscillations has been obtained. The 
Kamiokande II and 1MB collaborations observed an anomaly in the contained 
neutrino induced events. Both groups measure the ratio of muons to electrons and find 



^ ' it smaller than what is predicted by several independent theoretical calculations [Q. The 

events observed in these experiments have charged lepton energies in the range from a 
^ ■ few hundred MeV to 1.2 GeV. A possible interpretation of the effect has been given in 

^ , terms of ^ oscillations, where can be Uf,, z/^ or a sterile neutrino (z/g), with 

typical values for the oscillation parameters Am^ ~ 10^^ -t- 10~^ eV^ and sin^ 26 > 0.5 (a 
summary is contained in ref. 0). For this range of parameters, the flux of upward-going 
muons could also be observably reduced. However, the 1MB and Baksan experiments did 
not observe any such reduction |^ thus setting rather stringent limits to the allowed 
region in the (Am^, sin^ 29) plane. 

Neutrino oscillations (in vacuum ^ or in solar matter [^]) are also the most popular 
explanation for the solar neutrino deficit. In this case the u^, must oscillate into a neutrino 
of different flavour. However, both problems cannot be solved through the same Ue ^ 
oscillations, since the ranges of Am^ required in the two cases do not overlap. Thus, if 
the solar neutrino problem is solved through z/g ^ oscillations, then the atmospheric 
z/^'s must oscillate into something else. The simplest and often made hypothesis is that 
the contained events anomaly is explained through the z/^ oscillations. In this case, 
since both neutrino flavours have identical interactions with matter, the oscillations will 
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not be affected by the presence of tlie matter in tlie eartli. If, liowever, tlie initial z/^ 
oscillates to z/g or to a sterile neutrino, Ug, the matter effects can be relevant. 

In this letter we shall discuss in detail the various scenarios for atmospheric neutrino 
oscillations and show that indeed the matter effects are important, in that the allowed 



parameter region for both i/^ « 
compared to the z/„ i/^ case. 



and z/,, 



oscillations is appreciably different as 



The evolution equation of a system of two neutrino species {ui, uj) in matter is 
.d 
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Here i,j = e,/x,r or s where Vg is a sterile neutrino, i ^ 6 is the z/j — z/,- mixing angle 



in vacuum (0 < ^ < 7r/4), Am^ = m\ — m 
component of the mass eigenstate z/i, and 
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Veit) 



Aim? 



Ne{t) - 



Vi,{t) = Vr(t) 



_ 2V2EG1 

Avn} 

Vs{t) = 



Nn{t) 



(2) 



(3) 
(4) 



where and Nn are the electron and neutron number densities and Gp is the Fermi 
constant. The evolution of the antineutrino system is described by the same equation 
with Vij{t) substituted by —Vij{t). 

In matter of constant density the probability of Ui ^ Uj oscillations has the same form 
as in vacuum with the amplitude sin^ 26 and oscillation length i = AnE/ Arn^ substituted 
by 

8111^26 
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It can be seen from eq.(^ that matter can either enhance or suppress oscillations de 
pending on the relative signs and magnitudes of {vi — vj 
changes its sign with Am^. If the condition 



and cos 26. Note that (vj — Vj 



cos 261 
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is satisfied, the amplitude of oscillations in matter sin^ 26m becomes equal to unity, and 
the oscillations are resonantly enhanced (the MSW effect). The probability of the z/j uj 
transition will be large provided matter density varies slowly enough along the neutrino 
trajectory, i.e. the adiabaticity condition is fulfilled. In matter with uniform (or nearly 
uniform) density the transition probability will crucially depend on the thickness of the 



2 



matter slab. In general one needs to integrate numerically the evolution equation (|I|) 
along the neutrino path. 

The effect of matter on sin^ 29m in the case of z/^ ^ z/g oscillations is illustrated in fig. 
1, where it is plotted as a function of Am^/E, assuming a density p = 5 gcm~^, and an 
electron fraction Ne/{Np + Nn) = 1/2 (this corresponds to approximately average values 
in the earth). We can distinguish three regions: if Am'^/E > 10~^ eV^/GeV, the matter 
effects are negligible; if Am^/E < 10"'^ eV^/GeV, the matter effect supresses strongly the 
oscillation probabilities, and mixing in matter almost vanishes; in the transition region 
for Am^ > (Am^ < 0) the neutrino (antineutrino) mixing parameter passes through a 
resonance sin^(26'm) = 1 of width proportional to tan 26*, while the antineutrino (neutrino) 
effective mixing parameter goes to zero monotonically as the energy increases. In the case 
of maximal mixing in vacuum the oscillation probabilities of neutrinos and antineutrinos 
are equal and the sign of Am^ is irrelevant. 

The case of z/^ ^ oscillations can be calculated as before with the substitution 
Ne{t) —Nn{t)/2. The change of sign has the consequence that now for Am^ > 
(Am^ < 0) it is the antineutrino (neutrino) that will pass through the resonance. Since in 
the earth Nn ^ Np = N^,, fig. 1 can also be interpreted as describing the dependence on 
Am^/E of the mixing parameter sin^(2^m) in the case of z/^ ^ oscillations in matter 
with a density of 10 gcm~^, interchanging the neutrino and antineutrino curves. 

Atmospheric neutrinos are produced in the hadronic shower induced by primary cosmic 
rays in the earth's atmosphere. These neutrinos can be observed directly in large mass 
underground detectors by means of their charged- current interactions. "Contained events" 
are those where the neutrino-nucleus interaction vertex is located inside the detector and 
all final state particles do not get out from it. The average energy of the neutrinos that 
give rise to these events is a few hundred MeV. 

Muon neutrinos can also be detected indirectly observing the muons that they have 
produced in the material surrounding the detector. To reduce the background from at- 
mospheric muons, only upward-going neutrino-induced muons are usually considered. A 
rough estimate of the energy spectrum of the upward-going muons has been obtained 
dividing them in two categories, passing (those of highest energy) and stopping muons. 

The flux of upward-going muons can be calculated as 

(t>^^{E,,cos^) = dE, 0,^(^^)(E„cos^) (^^.^^^ 

where (l)^^(i!^){E^,cosip) is the z/^ (u^) spectrum at nadir angle and dn^^ij^ -^^^^/dE^ 
is the number of p^ with energy between E^ and Ep_ + dE^ produced by a neutrino 
(antineutrino) of energy E^: 

^^^^^^^^{E,;E^) = NArdX f"" dEo^^^{Eo,E^)5[E,-R-\R{Eo)-X)] (9) 
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In this equation, the first integral is over the position of neutrino absorption {X is in 
gcm~^, X = corresponds to the detector); the second integral is over the muon energy 
at the production point. The delta function expresses the fact that the muon created with 
initial energy Eq at the point X must reach the detector with energy E^. In the argument 
of the delta function, R{E) is the range in rock of a muon of energy E, and R~^{X) is 
the inverse function, giving the initial energy of a muon with range X. We are assuming 
that fluctuations in the energy loss are negligible, and that the energy-range relation is 
well defined. This is a good approximation in this range of muon energy (see ref. [1^ for 
a discussion). The neutrino cross section, expressed in terms of the the usual kinematical 
variables y = 1 — E^/E^, and x = Q'^/{2mj^E^y), can be calculated integrating over x 
the following expression: 



dxdy 



[x, y, 
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x[qd{x,Q^) + il-yyUx,Q^)] (10) 



In eq. ([10|) Gp is the Fermi constant, Mw is the W boson mass, m^r is the nucleon mass, 
and qd,u (jld u ) ^'^^ the distributions of quarks (antiquarks) of down-like and up-like type 
in the target nucleon. The antineutrino cross section is obtained interchanging q ^ q. 

Eq. d^) assumes that the neutrino flux is unmodifed in the passage through the earth. 
To take into account the effects of oscillations we have to make the substitution 



[1 - Pu, 



for I/r 



Ur or Us, or 



[1 - Pu, 



+ Pu. 



;i2) 



in the case of z/^ oscillations, and analogously for antineutrinos. 

Strictly speaking, the transition probability Py^^^^ depends on the oscillation param- 
eters Am^ and sin^ 26, neutrino energy E^, nadir angle ip, and also on the exact positions 
of creation and absorption points of neutrinos. The neutrinos are produced in a layer of 
the atmosphere with a thickness of approximately ~ 10 km, and the upward-going muons 
are created in a layer of thickness ~ 0.1 km below the detector. We will be discussing a 
range of Am? / E such that £ > 100 km and therefore it is a good approximation to neglect 
the dependence on the exact position of creation and absorption of the neutrinos, and 
assume that all neutrinos are generated in an infinitely thin spherical layer of atmosphere 
at the surface of the earth and that the muons are also produced in a thin layer of material 
below the detector. In this approximation the <-> v^- transition probability takes the 
form: 



P„ 



E 



- sm' 
2 
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1 — cos 



Am cos ip 
E 



(13) 



The probability of z/^ 



transition is equal to that in the neutrino case. 



The probabilities of the neutrino (and antineutrino) transitions z/^ z/g and z/^ ^ z/g 
have been calculated by integrating numerically the evolution equation ([^) using the 
distributions of Ne{r) and Nn{r) obtained from the matter density profile of the earth 
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taken from ref. |TT[ and assuming Nn ^ Np = Ne, which is quite a good approximation 



in the interior of the earth. 

Some effects of neutrino oscillations on the upward-going muon flux are described 
below, (i) The total rate is reduced with respect to expectations based on the assumption 
of no oscillations. To detect this effect one needs to control the absolute normalization of 
the calculated flux, (ii) The energy spectrum is distorted, because low energy neutrinos 
have a shorter oscillation length. The shape of the neutrino spectrum, after the crossing 
of the earth, is reflected in the energy spectrum of the upward-going muons. The ratio 
of stopping/passing muons in an underground detector is therefore a useful tool in the 
search for neutrino oscillations. In this measurement one needs only to control the shape 
(and not the normalization) of the calculated upward-going muon flux. A large part of 
the uncertainties in the calculations cancel, and the measurement is thus dominated by 
statistical errors, (iii) The nadir angle distribution is also modified, because longer path 
lengths correspond to more vertical directions. 

In our numerical work we have defined as "stopping muons" those with energy (1.25 
GeV < < 2.5 GeV), and "passing muons" those with (i?^ > 2.5 GeV). Integrating over 
the entire downward hemisphere the muon flux given in eq. (§) we obtain the integrated 
fluxes 

rl ,2.5(oo) 

$f(p) = 27r/ dcosi) / dE^ (l)^±{E^, cos ip). (14) 

•J '^1.25(2.5) 

In general the oscillation probabilities of neutrinos and antineutrinos are different, and 
therefore we compute separately the fluxes of positive and negative muons. However the 
existing detectors are unable to measure the charge of the detected particles, and so we 
then add together the calculated /i^ fluxes. The total flux of upward-going muons is 
$ = + $p = $+ + $7 + $+ + ^p. 

In the calculation of the upward-going muon fluxes in the absence of oscillations, 
one needs to specify the neutrino fluxes, the nucleon structure functions (needed in the 
calculation of the cross section), and the muon range. The imperfect knowledge of the 
first and second item introduce a significant systematic uncertainty. Using the (z/^ + V^) 
flux calculated by Volkova |]12|, with the u/v ratio of Lipari |]l3l, the structure functions 



given by Diemoz et al. , and the muon range taken from the report of Lohmann et al. 



[1151 , the calculated flux of upward-going muons is: 



27r 

(0.70 of the flux is due to negative muons), and 



-^ = 0.185 (16) 



(0.67 of the stopping muons are negative). The median energy of neutrinos giving rise to 
stopping (passing) muons is 5.5 (86) GeV. A different choice of the input in equation (H) 
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would modify these results, in particular there is a large sensitivity to the neutrino fluxes. 
Using the neutrino fluxes of Butkevich et al. |jl6|, results in a total upward-going muon 



flux larger than ([15]) by 11%. In this case the stopping/passing ratio is 0.189, a difference 
of less than 3%. Using the (z/^ + V^) fluxes of Mitsui fl^ would result in a total flux 6% 
(5.5%) larger than (|T5]) and a ratio of stopping to passing of 0.191 (0.190), with the v /v 
ratio of Butkevich (Lipari). The calculations with the structure functions of Eichten et 



al. []T8l result in a rate of muons smaller by 1% for set 1, and larger by 2.5% for set 2. 

The calculated upward-going muon flux (|1^ is implicitly a function of the oscillation 
parameters. Given the flavour of the neutrino mixed with the v^, each choice of a pair of 
parameters (Am^, sin^ 26') will result in a different flux. As an illustration, in flg. 2 we 
show the calculated $s/ $p ratio as a function of Am^ assuming maximal mixing (sin^ 26 = 
1). It is clear that the matter effect is important and that the curves corresponding to Ur, 
Uf, and Ug differ from each other. The qualitative behaviour of the curves can be simply 
understood. For Am^ < 10~^ eV^, the oscillation length even of the softest neutrinos 
that produce detectable muons is longer than the earth's radius, and there is no visible 
effect. With increasing Am^ the low energy neutrinos begin to oscillate, the spectrum 
is distorted and $<i/$p decreases. For Am^ > 1 eV^, all relevant neutrino energies are 
equally affected and, averaging over the very rapid oscillations, the shape of the spectrum 
is again the same as in the no-oscillation case. In the case of z/^ z/e oscillations we 
have to consider the presence of the I'eij'e) directly produced in meson decay. These 
neutrinos have a much steeper spectrum than the v^{v^). For large Am? averaging over 
rapid undetectable oscillations we have: (p^^ — > (1 — 0.5 sin^ 20)(l)y^ + 0.5 sin^ 26 (p^^. This 
yields a distortion of the spectrum with a relative excess of low energy neutrinos. 

Measured values of the upward-going muon flux (averaged over 2tt of solid angle), in 
units 10~^^ (cm^ s sr)~^, are: (2.77 ± 0.17) with a threshold of 1 GeV from the Baksan 
group 0, (2.26 ± 0.17) with a threshold of 2 GeV from the 1MB experiment [|l|], (2.04 ± 



0.13) with a threshold of approximately 3 GeV from the Kamiokande experiment The 
three measurements, after corrections for the different thresholds, are in agreement with 
each other, and very close to our calculated value (|T5p. To obtain an allowed region in the 
(Am^, sin^ 26') plane would require a detailed discussion of the systematic uncertainties 
of the theoretical calculation. The 1MB group |^ derives a rather stringent limit from 
their measurement of the total rate of upward-going muons. A more critical view of the 



systematic uncertainties, however, would result in much less stringent limit Rather 
arbitrarily, we have chosen as a criterion to deflne the region of parameter space that can 
be excluded at 90% c.l. because of the measurement of the total rate the condition 

$ < 0.75 $0 , (17) 

$0 being the average flux ([T5|) calculated in the absence of oscillations. The limiting 
curves calculated using this criterion (curves of type A) for different types of oscillations 
are shown in flg. 3 for the ^ and z/^ ^ cases, and in flg. 4 for the z/^ v^, case. 

The 1MB collaboration has measured |P a stopping/passing ratio 0.160 ± 0.019, in 
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good agreement with their detailed Monte Carlo predictions. Our calculated value of 
0.185 agrees reasonably well with the 1MB result, considering the approximations we have 
used[|. In this case we have used as a criterion (90% c.l.) for oscillations the condition: 

1^ < 0-8 (1^) (18) 

where again the subscript indicates the ratio of fluxes calculated in the absence of neutrino 
oscillations. Curves calculated with this criterion (curves of type B) are shown in fig. 3 
for the ^ and ^ cases. For the ^ z/g oscillations the condition (|1^) is 
never satisfied. In fig. 4 we show the region of parameter space that could be excluded 
with the requirement $s/'^'p ^ 0.9 ($s/$p)o, more demanding from the point of view of 
statistical accuracy. 

The situation about experimentally allowed values of oscillation parameters in the case 
of ^ Vj. and <->■ Vg oscillations is summarized in fig. 3. In this figure three limiting 
curves are the results of experiments that do not involve upward-going muons. Curve (a) 
is obtained from accelerator experiments and the allowed region is below the curve. 



Curve (b) is obtained from the result of Frejus |^ on the e//i ratio of contained events; 
this result is consistent with the no-oscillation hypothesis, and the allowed region is to 
the left of the curve. Curve (c) is obtained from the observation of an anomaly in the 
same e//i ratio by Kamiokande and 1MB 0, this is a 'positive result' and the allowed 
region is to the right of the curve. These three limits apply equally to ^ and 
^ oscillations which differ from each other only because of the matter effects. 
These effects are obviously irrelevant to the accelerator limit, and almost so also for (b) 
and (c) curves, although approximately half of the contained events observed in 1MB 
and Kamiokande are produced by neutrinos that have penetrated through the earth. In 
fact, these neutrinos have energies Ey < 1.2 GeV, with a rapidly falling spectrum. On 
curves (b) and (c) the absolute value of the squared mass difference is always larger than 
10~^ eV^. For matter effects to be significant one needs 

Am,^ eV^ 

< V2GfNaP = 0.758 ■ 10-V(g cm'^') ^ (19) 

The maximum density of the earth is p ~ 12.5 gcm~^, thereforefor contained events 
matter effects are practically negligible and their inclusion would only slightly modify the 
shape of curve (c) in the region of lowest |Am^|. 

Matter effects are on the contrary significant for upward-going muons, the reason 
being that the neutrino energies involved are one to two orders of magnitude larger than 
those for contained events. 



^In the 1MB detector the threshold energy for muon detection was approximately 1.8 GeV in the 
initial part of the data taking (IMB~1,2), and 1.0 GeV for the final part (IMB~3). The criterion for 
stopping muons is that there is essentially no light for the last 5 meters of the projected track length 
inside the detector. Considering the geometrical dimension of the detector (18x17x22.5 m^) , this sets 
a maximum energy for stopping muon of ~ 2.5 GeV. 
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The limits obtained from the total flux of upward-going muons using the criterion ( ]T7| ) 
are shown in fig. 3 by the curves Ar, As+ and A^-, the subscript indicating the flavour of 
the neutrino mixed with the z/,-. The case Ug is described by two curves because we need 
to consider the sign of Am^. The limits obtained from the stopping/passing ratio using 
the criterion ( [T8| ) are shown in the same figure by the curves B^-, Bs+ and Bs-. 

Comparing the three type A curves we can make the following remarks. For maximal 
mixing the limits on |Am^| for z/^ Ug oscillations is a factor 2.6 less stringent than for 
^T- If sin^ 26' = 1 the matter can only suppress the oscillations, in fact z/^'s with 
\Am'^\/E < 2-10~^ eV^/GeV have their mixing with Ug strongly suppressed (see fig. 1) and 
do not oscillate. For maximal mixing the sign of Am^ is irrelevant and therefore the curves 
As+ and Ag- end at the same point. With decreasing mixing the two curves separate, 
the limit for negative Am^ being stronger. When Am^ < (Am^ > 0) for oscillations of 
neutrinos (antineutrinos) the MSW resonant enhancement will occur. Since z/^'s are more 
abundant (and have a larger cross section) than z7^'s, the same values of sin^ 29 and | Am^| 
will result in a stronger suppression of the upward-going muon flux if Am^ is negative 
and the MSW resonance is relevant for neutrinos. For |Am^| > 1 eV^ the three curves At-, 
Ag+ and Ag- almost coincide; this is again a reflection of the fact that for large \Am'^\/E 
matter effects are not significant. 

Very similar considerations can be made about the curves Br, Bg+ and Bg-. For 
maximal mixing, the region of |Am^| that can be excluded in the case of z/^ ^ Ug os- 
cillations is moved to values of |Am^| larger by a factor 2.8 than in the Uj. case. 
For oscillations into sterile neutrinos, when the mixing is smaller than unity, the limit 
that applies for Am^ < is stronger than for the other case. This is again because when 
the MSW resonance occurs for neutrinos (antineutrinos) the sensitivity to oscillations is 
stronger (weaker). A detailed analysis must also take into account the fact that when the 
neutrinos resonate the oscillation length passes trough a maximum. This is the reason 
why the limit Bg+ is stronger than the limit Bg- in a limited region of parameter space. 
For maximal mixing the curves Bg+ and Bg- end in the same points. 

The most significant difference between the limits for z/^ ^ and z/^ ^ Ug oscillations 
is in the region of low squared mass differences. A region of parameter space |Am^| = 
(3 -h 7) ■ 10"'' eV^ and sin^26' > 0.7 is excluded for z/^ but not for Ug oscillations. This 
region is however already excluded by curve (c), provided that the anomaly in the e/fi 
ratio of contained events is due to neutrino oscillations. As shown in fig. 3, for both types 
of oscillations there is a region of parameter space which is compatible with all existing 
experimental measurements. The range of allowed values for the parameters is in both 
cases roughly: 0.4 < sin2 2^ < 0.7 and 2 ■ lO'^ (eV^) < \Am^\ < 0.4 (eV^). The allowed 
region for Ug is somewhat smaller but contains also a small part (for large positive Am^) 
which is not allowed for z/^. In future, with more data, the sensitivity of the curves of type 
B (which is determined by the statistical errors) will improve, exploring the low |Am^| 
part of the allowed region. To improve the sensitivity of the curves of type A, one needs 
rather to reduce the theoretical systematic uncertainties. If these could be reduced to 
the level of 15% (at 90% c.l.) the entire allowed region of parameter space would then 
be explored by measurements of upward-going muons and one would be able to either 
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confirm or disprove the neutrino oscillations as a solution to the atmospheric neutrino 
puzzle. 



The situation about the experimentally allowed values of the oscillation parameters in 
the case of i/^ v^. oscillations is summarized in fig. 4. The results from the analysis of 
solar neutrino experiments refer to lower values of |Am^| and are not shown. As in fig. 
3, three limiting curves are the results of experiments that do not involve upward-going 
muons. Curve (a) is the limit obtained from reactor experiments |2i|, curve (b) and (c) 



are obtained from measurements of the e//i ratio of contained events in the Frejus ||2^ 
and Kamiokande and 1MB experiments |Q |^ . As before, curve (b) excludes the region to 
its right, curve (c) the region to its left. 

The limits obtained from the measurement of the total upward-going muon flux ac- 
cording to criterion is shown by curves and A^- , the two curves referring to the 
sign of Am^ as before. The region excluded by the curves of type A is well inside the 
region of parameter space already excluded by the Gosgen reactor experiment. Matter 
suppresses the oscillations more strongly than in the z/^ <-> case, since the difference in 
the effective potentials is twice as large now, see eqs. We also have to take into 

account the fact that in cosmic ray showers a Ve^ei flux is produced as well. Both effects 
reduce the sensitivity of the measurement to z/^ ^ z/e oscillations. 

As already mentioned, the condition (|1^) is never satisfied, and therefore no limit 
can be obtained from the stopping/passing ratio. In the case of maximal mixing, $s/$p 
reaches (for |Am^| = 6 ■ 10~^eV^) a minimum value of 0.150, only 19% smaller than 
the value calculated in the absence of oscillations. Curves and B^- are calculated 
with the more demanding criterion < 0.9 ($<j/^p)o- They are an indication of the 

sensitivity that could be obtained with a sample of data approximately four times the one 
collected by 1MB. 

As shown in fig. 4, a small region of parameter space 0.35 < sin^20 < 0.7 and 
4 ■ 10~^ (sV^) ^ |Am^| < 2 ■ 10^^ (eV^) is compatible with all existing experimental 
measurements. Future experiments on upward muon fluxes will be able to explore only a 
part of this region. 

As we have seen, the oscillations of z/^ into a sterile neutrino state z/g can be strongly 
affected by the matter of the earth. Although the existence of a sterile neutrino does 
not contradict any laboratory data, it can be in conflict with cosmological considerations. 
Namely, for large enough values of the mixing angle and Am^, z/^ ^ Vg oscillations 
can bring the sterile neutrinos into equilibrium with matter before the nucleosynthesis 
epoch thereby affecting the primordial ^He abundance in the universe. The analysis 



performed in [EH] sets the maximum allowed number of "light neutrinos" N^, to 3.4. The 



limit derived in |^ not to have N^, > 3.4 for oscillations between sterile and muon 
neutrinos is Am? < 8 ■ 10~^ eV^ for maximal mixing. With the values we obtained in our 
analysis this bound would be violated, and we would have = 4. The limit given in ||2^ 



is not however universally accepted and is still a point of debate , so that we believe 
that z/^ ^ Us oscillations may still provide a viable solution to the atmospheric neutrino 
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problem. 

As to the Un ^ Vf, oscillations hypothesis for atmospheric neutrinos, it should be 
noted that this assumption does not contradict the idea of neutrino oscillations being also 
responsible for the observed solar neutrino deficit. The latter can be accounted for through 
the Vf. Vj. oscillations provided the neutrino masses obey the conditions ~ rrty^ -C 
irty^ or m^^ -C rrty^ ^ ^v^- We would like to stress that although many models (including 
those implementing the see-saw mechanism of neutrino mass generation) predict the 



neutrino mass hierarchy rriy^ <^ rriy^ <^ f^v-ri models with quite different hierarchies also 
exist. Moreover, there are absolutely no experimental indications in favour of the direct 
hierarchy. 

In summary, we have shown that the complex of data induced by atmospheric neutrinos 
("contained" and "upward-going muons") may still be described in terms of neutrino 
oscillations. The matter effects are important in the precise determination of the allowed 
parameter region. 

P.L. is grateful to T.K. Gaisser and T. Stanev for many useful conversations and for 



showing us an early version of ref. E.A. acknowledges fruitful discussions with Z. 

Berezhiani and A. Smirnov. 
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Figure Captions 



Fig. 1. Plot of sin^ 29m, the mixing parameter in matter for oscillations, 
as a function of Am'^/E, for sin^ 2^ = 1,0.5,0.05, assuming a matter density p — 
5 gcm~^ and an electron fraction Yg — 0.5. Solid curves are for neutrinos, dashed 

curves for antineutrinos. If Am^ < the neutrino and antineutrino curves must be 
interchanged. The same curves can also be interpreted as the mixing parameter in 
matter for the z/^ transition, assuming a matter density p' = 10 gcm~^ and 

interchanging the neutrino and antineutrino curves. 

Fig. 2. Plot of the $,5/$p ratio as a function of |Am^|, assuming maximal mixing 
in vacuum. The different curves are for ^ i^t, i^/j. ^ i^e, and ^ Vg oscillations. 
We also show the curve calculated for the ^ transitions neglecting the Ve^e) 
created directly in meson decays. 

Fig. 3. Limits for the oscillation parameters Am^ and sin^ 26 in the case of Vr 
and <-> mixing. The curves At, As+ and ^4^- are limits obtained from the 

measurements of the total flux of upward-going muons. The curves 5^, and Bg- 
are obtained from the measurements of the stopping/passing ratio. Also plotted are 
the 90% c.l. limits from accelerator experiments (a), and the from the measurement 
of the e//i ratio of contained events in the Frejus (b) and Kamiokande experiments 
(c). 

Fig. 4. Limits for the oscillation parameters Am^ and sin^ 29 in the case of i/^ 
mixing. The curves Ag+ and A^- are obtained from the measurements of the total 
flux of upward-going muons. The curves Bg+ and B^- are in this case estimates 
of the sensitivity of future measurements of the stopping/passing ratio (sec text). 
Also plotted are the 90% c.l. limits from the Gosgen reactor experiment (a), and 
from the e/ p ratio of contained events in Frejus (b), and Kamiokande (c). 
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p = 5 g cm-^ Yg-l/S. [p=10 g cm-=^ (i/g ^ i/g)] 
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Figure 1 




Figure 2 




a: CDH$ 
b: Frejus 
c: Kamioka 

A: 0tot 
B: 0g/0p 

J I I I I I I I I 



I I I I I I I I I I I I 



0.2 0.4 0.6 0.8 



Sin''(20) 



Figure 3 
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